Posture information based pedestrian detection and pedestrian collision prevention apparatus and method

ABSTRACT

An apparatus for activating a pedestrian detection and collision mitigation system (PDCMS) of a vehicle includes: a front detection sensor detecting a presence of a pedestrian on a driving lane of the vehicle, gaze information of the pedestrian, and a distance and a relative speed between the pedestrian and the vehicle; a vehicle sensor detecting at least any one of a speed, an acceleration, a steering angle, a steering angular velocity, or a pressure of a master cylinder of the vehicle; an electronic control unit activating a PDCMS function based on information detected by the front detection sensor and the vehicle sensor; and a warning unit operated to inform a driver of a collision of the pedestrian with the vehicle by a control of the electronic control unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is the continuation application of U.S. patentapplication Ser. No. 15/833,667, filed on Dec. 6, 2017, now allowed,which claims the benefit of priority to Korean Patent Application No.10-2016-0184290, filed on Dec. 30, 2016, the disclosure of which isincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a pedestrian detection and apedestrian collision prevention apparatus and method, and moreparticularly, to an apparatus and a method for activating a pedestriandetection and collision mitigation system (PDCMS) of a vehicle capableof recognizing a pedestrian by analyzing a posture of the pedestrian inan image in front of the vehicle and to protect a pedestrian byactivating a PDCMS function when an accident is highly likely to occurby analyzing gaze information of the pedestrian.

BACKGROUND

Recently, advanced driver assistance systems (ADAS) have been developedto assist driving of a driver. The ADAS has multiple sub-technologycategories. Among those, the ADAS includes a pedestrian detection andcollision mitigation system (PDCMS).

The PDCMS is a technology that warns a driver of a pedestrian collisionwhen a collision of a pedestrian with a vehicle is expected andautomatically activates an emergency brake.

Lethality and injury rates of pedestrian-related traffic accidents areso high, which leads to a lot of life loss. The PDCMS system may helpreduce a speed of the vehicle against inevitable pedestrian collisions,thereby alleviating pedestrian impacts and reducing the lethality andthe injury rates.

Therefore, a technology development for specific application of thePDCMS has been required.

SUMMARY

An object of the present disclosure is to an apparatus for activating apedestrian detection and collision mitigation system (PDCMS) including afront detection sensor capable of measuring a presence of a pedestrianand a distance and a relative speed between a vehicle and a pedestrianwith higher accuracy, based on posture information of a pedestrian in animage in front of the vehicle.

Another object of the present disclosure is to provide a system for moresafely protecting a pedestrian by accurately activating a PDCMSfunction.

Other objects and advantages of the present disclosure can be understoodby the following description, and become apparent with reference to theembodiments of the present disclosure. Also, it is obvious to thoseskilled in the art to which the present disclosure pertains that theobjects and advantages of the present disclosure can be realized by themeans as claimed and combinations thereof.

In accordance with one aspect of the present disclosure, an apparatusfor activating a pedestrian detection and collision mitigation system(PDCMS) of a vehicle includes: a front detection sensor detecting apresence of a pedestrian on a driving lane of the vehicle, gazeinformation of the pedestrian, and a distance and a relative speedbetween the pedestrian and the vehicle; a vehicle sensor detecting atleast any one of a speed, an acceleration, a steering angle, a steeringangular velocity, or a pressure of a master cylinder of the vehicle; anelectronic control unit activating a PDCMS function based on informationdetected by the front detection sensor and the vehicle sensor; and awarning unit operated to inform a driver of a collision of thepedestrian with the vehicle by a control of the electronic control unit,in which the front detection sensor may include: an image inputprocessor acquiring an image in front of the vehicle; an ROI settingprocessor setting a region of interest (ROI) corresponding to a size ofan object in the image; a candidate extraction processor extracting apedestrian candidate based on an operation of the object in the set ROI;and a pedestrian determination processor including a database andcomparing the extracted pedestrian candidate with a posture of a samplepedestrian in the database to identify whether the extracted pedestriancandidate corresponds to the pedestrian, and the PDCMS function mayinclude an activation of an operation of the warning unit and anactivation of an operation of a brake regardless of whether the driveroperates the brake and the activation of the operation of the warningunit and the activation of the operation of the brake may be performedin order of the operation of the warning unit, a partial braking of thevehicle, and a full braking of the vehicle.

The pedestrian determination processor may include the database andstores, as the posture of the sample pedestrian, information associatedwith at least one of front, back, left, and right portions, an upperleft half of a body, and an upper right half of a body of the samplepedestrian in the database.

The pedestrian determination processor may compare the extractedpedestrian candidate with the front, back, left, and right portions, theupper left half of the body, or the upper right half of the body of thesample pedestrian in the database in a cascade manner.

The front detection sensor may further include a pedestrian trackingprocessor tracking the pedestrian identified by the pedestriandetermination processor.

The pedestrian tracking processor may track the pedestrian based on thenumber of feature points of the pedestrian in the set ROI.

When a horizontal length of the entire face of the pedestrian is definedas x, a horizontal length from a left face contour to a left eye of thepedestrian is defined as x₁, a horizontal length from a right facecontour to a right eye of the pedestrian is defined as x₂, and ahorizontal length between the left eye and the right eye of thepedestrian is defined as x₃, the gaze information of the pedestrian maycorrespond to a front or a diagonal if |(x₁−x₂)/x|<a, a rear ifx₁=x₂=x₃=0, and a side if |(x₁−x₂)/x|≥a or x₁=0 or x₂=0, and the a maybe selected in a range between 0.65 and 0.95.

The electronic control unit may perform the activation of the operationof the warning unit and the activation of the operation of the brake bydelaying the activation of the operation of the warning unit and theactivation of the operation of the brake by a predetermined time whenthe gaze information of the pedestrian is the front or the diagonal,compared to when the gaze information of the pedestrian is the rear orthe side.

The vehicle sensor may further include at least any one of a rainsensor, a temperature sensor, and an illumination sensor.

The electronic control unit may perform the activation of the operationof the brake so that the speed of the vehicle is reduced to at least apredetermined speed or more from time when the operation of the brake isactivated to time when the collision of the pedestrian with the vehicleoccurs.

The electronic control unit may permit the driver to operate the brakefor a maximum possible deceleration even after the activation of theoperation of the brake starts.

The electronic control unit may control the warning unit to inform thedriver that the PDCMS function is in an available state.

The warning unit may include a display unit visually informing thecollision of the pedestrian with the vehicle or a speaker unit audiblyinforming the collision of the pedestrian with the vehicle.

The PDCMS function may further include an operation of a rear brakelamp.

The PDCMS function may further include an operation of an electricalstability control (ESC).

In accordance with another aspect of the present disclosure, a methodfor activating a pedestrian detection and collision mitigation system(PDCMS) of a vehicle includes: acquiring an image in front of thevehicle; setting a region of interest (ROI) corresponding to a size ofan object in the image; extracting a pedestrian candidate based on anoperation of the object in the set ROI; including a database andcomparing the extracted pedestrian candidate with a posture of a samplepedestrian in the database to identify whether the extracted pedestriancandidate corresponds to the pedestrian on a driving lane of thevehicle, and detecting, by a front detection sensor, gaze information ofthe pedestrian and a distance and a relative speed between thepedestrian and the vehicle; detecting, by a vehicle sensor, at least anyone of a speed, an acceleration, a steering angle, a steering angularvelocity, or a pressure of a master cylinder of the vehicle; andactivating a PDCMS function based on information detected by the frontdetection sensor and the vehicle sensor, in which the PDCMS function mayinclude an activation of an operation of a warning unit operated toinform a driver of a collision of the pedestrian with the vehicle and anactivation of an operation of a brake regardless of whether the driveroperates the brake and the activation of the operation of the warningunit and the activation of the operation of the brake may be performedin order of the operation of the warning unit, a partial braking of thevehicle, and a full braking of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and other advantages of thepresent disclosure will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a diagram illustrating a schematic concept of a PDCMS;

FIG. 2 is a block diagram illustrating a change in a PDCMS stateaccording to a vehicle;

FIG. 3 is a block diagram schematically illustrating an apparatus foractivating a PDCMS of a vehicle according to an embodiment of thepresent disclosure;

FIG. 4 is a configuration diagram of a front detection sensor accordingto an embodiment of the present disclosure;

FIG. 5 is an exemplified diagram illustrating ROI setting according toan embodiment of the present disclosure;

FIG. 6 is an exemplified view for identifying gaze information of thefront detection sensor according to an embodiment of the presentdisclosure;

FIG. 7 is a diagram illustrating a concept of a pedestrian moving speed;

FIG. 8 is a diagram illustrating an example of a mapping table foractivating a PDCMS function according to the embodiment of the presentdisclosure;

FIG. 9 is a diagram illustrating an example of the operation of thePDCMS function according to the embodiment of the present disclosure;

FIGS. 10A-10C are exemplified views illustrating the operation of thePDCMS function when the gaze information is the front according to theembodiment of the present disclosure;

FIGS. 11A-11D are exemplified views illustrating the operation of thePDCMS function when the gaze information is the diagonal according tothe embodiment of the present disclosure;

FIGS. 12A-12C are exemplified views illustrating the operation of thePDCMS function when the gaze information is the rear according to theembodiment of the present disclosure;

FIGS. 13A-13C are exemplified views illustrating the operation of thePDCMS function when the gaze information is the side according to theembodiment of the present disclosure; and

FIG. 14 is a flow chart illustrating a flow of a method for activating aPDCMS function according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings so thatthose skilled in the art may easily practice the present disclosure. Asthose skilled in the art would realize, the described embodiments may bemodified in various different ways, all without departing from thespirit or scope of the present disclosure.

A part irrelevant to the description will be omitted to clearly describethe present disclosure, and the same elements will be designated by thesame reference numerals throughout the specification.

Throughout the present specification, when any one part is referred toas being “connected to” another part, it means that any one part andanother part are “directly connected to” each other or are “electricallyconnected to” each other with the other part interposed therebetween.Further, unless explicitly described to the contrary, “comprising” anycomponents will be understood to imply the inclusion of other componentsrather than the exclusion of any other components.

The mention that any portion is present “over” another portion meansthat any portion may be directly formed on another portion or a thirdportion may be interposed between one portion and another portion. Incontrast, the mention that any portion is present “just over” anotherportion means that a third portion may not be interposed between oneportion and another portion.

Terms used throughout the specification, ‘first’, ‘second’, ‘third’,etc. may be used to describe various portions, components, regions,layers, and/or sections but are not limited thereto. These terms areused only to differentiate any portion, component, region, layer, orsection from other portions, components, regions, layers, or sections.Therefore, a first portion, component, region, layer, or section whichwill be described below may be mentioned as a second portion, component,region, layer, or section without departing from the scope of thepresent disclosure.

Terminologies used herein are to mention only a specific exemplaryembodiment, and does not limit the present disclosure. Singular formsused herein include plural forms as long as phrases do not clearlyindicate an opposite meaning. A term “including” used in the presentspecification concretely indicates specific properties, regions, integernumbers, steps, operations, elements, and/or components, and is not toexclude presence or addition of other specific properties, regions,integer numbers, steps, operations, elements, components, and/or a groupthereof.

The term expressing the relative space of “under”, “over”, and the likemay be used to more easily describe the relationship between otherportions of one portion which is illustrated in the drawings. The termsintend to include other meanings or operations of apparatuses which arebeing used along with the intended meaning in the drawings. For example,overturning the apparatus in the drawings, any portions described asbeing positioned “under” other portions will be described as beingpositioned “over” other portions. Therefore, the exemplified term“under” includes both of the up and down directions. An apparatus mayrotate by 90° or may rotate at different angles and the term expressinga relative space is interpreted accordingly.

All terms including technical terms and scientific terms used hereinhave the same meaning as the meaning generally understood by thoseskilled in the art to which the present disclosure pertains unlessdefined otherwise. Terms defined in a generally used dictionary areadditionally interpreted as having the meaning matched to the relatedart document and the currently disclosed contents and are notinterpreted as ideal or formal meaning unless defined.

FIG. 1 is a diagram illustrating a schematic concept of a PDCMS.

The PDCMS is a technology that warns a driver of a pedestrian collisionwhen a collision of a pedestrian with a vehicle is expected andautomatically activates an emergency brake.

Referring to FIG. 1, it is determined whether the PDCMS is operatedbased on an operation determination of a pedestrian and an operationdetermination of a vehicle. When the operation of the PDCMS isdetermined, the PDCMS function is performed by issuing a warning todriver and activating a vehicle control.

A system designer may design the PDCMS function to operate solely in therisk of collision of a pedestrian with a vehicle or may design the PDCMSfunction to operate in combination with other driving assistancesystems.

FIG. 2 is a block diagram illustrating a change in a PDCMS stateaccording to a vehicle.

In a PDCMS off state, no action is taken on the operation of thevehicle. The PDCMS off state is produced when an engine of a vehiclestalls.

In the PDCMS deactivation state, the apparatus for activating a PDCMSmonitors a speed of a vehicle and determine whether the PDCMS is in anappropriate state to activate. The PDCMS deactivation state is producedby turning on the engine in the PDCMS off state. Further, the PDCMSdeactivation state is produced even when the vehicle is in a state otherthan the conditions that the vehicle is activated from the PDCMSactivation state. For example, when the speed of the vehicle falls belowa predetermined value Vmin, the PDCMS deactivation state is produced.

The PDCMS activation state is produced when the speed of the vehicle isequal to or greater than the predetermined value Vmin and equal to orless than a predetermined value Vmax. To determine whether to operatethe PDCMS function in the PDCMS activation state, an operation of apedestrian and an operation of a vehicle are monitored. When theapparatus for activating a PDCMS determines that the PDCMS functionneeds to be operated, the PDCMS function starts. The PDCMS functionincludes a collision warning to a driver and an operation of anemergency brake or optionally includes braking actions by a driver.

FIG. 3 is a block diagram schematically illustrating an apparatus foractivating a PDCMS of a vehicle according to an embodiment of thepresent disclosure.

Referring to FIG. 3, an apparatus 100 for activating a PDCMS of avehicle according to an embodiment of the present disclosure includes afront detection sensor 200, a vehicle sensor 300, an electronic controlunit 400, and a warning unit 500.

The front detection sensor 200 may accurately recognize a pedestrian bysetting a region of interest (ROI) in an image in front of a vehicleacquired through a camera. A method for determining a pedestrian basedon the ROI setting of the front detection sensor 200 according to theembodiment of the present disclosure will be described in detail withreference to FIG. 4. The front detection sensor 200 may extractcharacteristics of obstacles detected in front of the vehicle toidentify objects and detect various objects such as vehicles on aroadside as well as pedestrians. The front detection sensor 200 maydetect even parts configuring a pedestrian as well as the overallappearance of the pedestrian to detect the pedestrian even when only apart of the pedestrian covered by various objects such as vehicles on aroadside is detected. Further, the front detection sensor 200 may detectgaze information of a pedestrian and a distance and a relative speedbetween a pedestrian and a vehicle when an object in front of thevehicle is determined as the pedestrian. The front detection sensor 200transmits the detected information on the pedestrian to the electroniccontrol unit 400.

The vehicle sensor 300 measures revolutions per minute (RPM) of avehicle wheel from a vehicle engine and calculates a driving speed of avehicle based on the known circumference of the wheel and the measuredRPM and time. Further, the vehicle sensor 300 may detect information ondriving conditions of a vehicle such as an acceleration, a steeringangle, a steering angular speed, and a pressure of a master cylinder.Further, the vehicle sensor 300 may also detect information on drivingenvironment of a vehicle by including a rain sensor, a temperaturesensor, an illuminance sensor, etc. The vehicle sensor 300 may transmitthe information on the detected driving conditions and drivingenvironment of the vehicle to the electronic control unit 400.

The electronic control unit 400 determines whether to operate the PDCMSfunction of the vehicle based on the information received from the frontdetection sensor 200 and the vehicle sensor 300. Specifically, theelectronic control unit 400 determines whether the conditions that thePDCMS function may be operated are satisfied by combining the pedestrianstate and the vehicle state. That is, the electronic control unit 400determines the risk of collision between a vehicle and a pedestrianusing a current position of the pedestrian, a current position of thevehicle, and speed information on the vehicle if it is determined thatan obstacle is the pedestrian. For example, if the distance between thepedestrian and the vehicle is below a predetermined distance and themotion direction of the pedestrian is the same as the movement directionof the vehicle, it is determined that the conditions that the PDCMSfunction may be operated are satisfied because the collision is highlylikely to occur and if the distance between the pedestrian and thevehicle is below a predetermined distance but the motion direction ofthe pedestrian differs from the movement direction of the vehicle, it isdetermined that the conditions that the PDCMS function may be operatedare not satisfied because the collision is less likely to occur.

Preferably, the electronic control unit 400 determines whether theconditions that the PDCMS function may be operated are satisfied basedon the mapping table. The mapping table will be described below withreference to FIG. 6.

If the electronic control unit 400 determines that the pedestrian stateand the vehicle state satisfy the conditions that the PDCMS function maystart, the PDCMS function of the vehicle is operated. The PDCMS functionincludes operating the warning unit 500 to warn the driver of thecollision of the pedestrian with the vehicle or operating the brakewithout the operation of the driver.

Warning the driver of the collision of the pedestrian with the vehicleis performed by operating the warning unit 500. The warning unit 500 isoperated by the control of the electronic control unit 400. The warningunit 500 may include a display unit or a speaker unit. The display unitincluded in the warning unit 500 may provide a driver with a visualwarning through a head-up display, a navigation display, etc. Thespeaker unit included in the warning unit 500 may provide a driver withan audible warning through an audio. The content of the warning that thewarning unit 500 performs is that there is a potential risk of collisionof the pedestrian with the vehicle since obstacles exist in the front ofa driving lane of the vehicle.

The activation of the operation of the brake regardless of whether thedriver operates the brake is performed only by the control of theelectronic control unit 400 without the operation of the driver. Theactivation of the operation of the brake is to automatically reduce therelative speed between the vehicle and the pedestrian if it is foundthat the pedestrian collision is just around the corner.

The activation of the operation of the brake is performed so that thespeed of the vehicle may be reduced to at least a predetermined speed ormore from the time when the operation of the brake is activated to thetime when the collision of the pedestrian with the vehicle occurs.Preferably, the predetermined speed may be 20 km/h.

Further, even after the activation of the operation of the brake starts,the driver manually operates the brake, thereby performing the maximumpossible deceleration. That is, the driver may manually operate thebrake so that the speed of the vehicle is reduced more than thepredetermined speed. For example, the driver may manually operate thebrake so that the speed of the vehicle is maximally decelerated to 20km/h or more that is the predetermined speed.

In addition, the electronic control unit 400 may inform a driver thatthe PDCMS function is in an available state. Specifically, theelectronic control unit 400 may control the warning unit 500 to informthe driver that the PDCMS function is in the available state through thedisplay unit or the speaker unit of the warning unit 500.

In addition, the PDCMS function may control an operation of a brake lampto prevent the potential risk of collision with the following vehicles.

In addition, the PDCMS function may further include an operation of anelectrical stability control (ESC). The ESC is an apparatus that allowsa vehicle itself to intervene in an operation of a brake in an emergencysituation such as an oversteer (when a vehicle enters inwardly beyond aturning radius of a road) or an understeer (when a vehicle deviatesoutwardly beyond the turning radius of the road) of a vehicle to therebyhelp a driver to escape from an emergency situation.

FIG. 4 is a configuration diagram of a front detection sensor accordingto an embodiment of the present disclosure.

Referring to FIG. 4, the front detection sensor 200 according to theembodiment of the present disclosure includes an image input processor210 a region of interest (ROI) setting processor 220, a candidateextraction processor 230, a pedestrian determination processor 240, anda pedestrian tracking processor 250.

Each unit configuring the front detection sensor 200 may be controlledby the electronic control unit 400.

The image input processor 210 receives an image acquired by a camerathat photographs a front of a vehicle. The ROI setting processor 220sets an ROI corresponding to a size of an object in an image in front ofa vehicle received through the image input processor 210. That is, theROI setting processor 220 uses a size of the image, installationenvironment information of the camera, specifications of the camera, andan actual size of a pedestrian to determine a floor position of apedestrian area in the image. Therefore, since the pedestrian may beidentified based on the number of pixels, the ROI setting processor 220may set the minimum and maximum size of the pedestrian from the floorposition of the pedestrian area to be an appropriate ROI.

FIG. 5 is an exemplified diagram illustrating ROI setting according toan embodiment of the present disclosure.

Referring to FIG. 5, the installation environment information of thefront detection sensor 200 may be expressed by the following Equation 1.

$\begin{matrix}{{{H_{P\_ I}(i)} = {\frac{\psi_{P}(i)}{\theta_{V}} \times H_{I}}}{{\psi_{P}(i)} = {{\varphi_{PT}(i)} - {\varphi_{PB}(i)}}}{{\varphi_{PT}(i)} = \left\{ {{\begin{matrix}{{\arctan \left( \frac{H_{P\_ R} - H_{C}}{D_{P}(i)} \right)} + {90{^\circ}}} & {{{if}\mspace{14mu} H_{C}} < H_{P\_ R}} \\{{90{^\circ}} - {\arctan \left( \frac{H_{C} - H_{P\_ R}}{D_{P}(i)} \right)}} & {otherwise}\end{matrix}{D_{P}(i)}} = {{{\tan \left( {\varphi_{PB}(i)} \right)} \times H_{C}{\varphi_{PB}(i)}} = {{{i \times \frac{\theta_{V}}{H_{I}}} + {\theta_{IB}\theta_{IB}}} = {\arctan \left( \frac{D_{IB}}{H_{C}} \right)}}}} \right.}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In the above Equation 1, the respective symbols have the followingmeanings.

i: Floor position of the pedestrian area in the image

H_(P_I): Size (height) of the pedestrian area in the image

ψ_(P): Angle in a vertical direction which is occupied by the pedestrian

H_(i): Size (height) of the input image

ϕ_(PT): Angle up to a crown of a pedestrian's head

ϕ_(PB): Angle up to a tip of a pedestrian's toe

H_(P_R): Size (height) of the pedestrian

D_(P): Distance up to the pedestrian

H_(C): Height at which the camera is installed

θ_(V): Angle in a vertical direction of the camera

θ_(IB): Angle up to an actual position corresponding to a starting pointof the image

D_(IB): Distance up to an actual position corresponding to the startingpoint of the image

The candidate extraction processor 230 extracts a pedestrian candidatebased on an operation of an object having the ROI set by the ROI settingprocessor 220.

The pedestrian determination processor 240 identifies the pedestrian bycomparing the pedestrian candidate extracted by the candidate extractionprocessor 230 with a posture of a sample pedestrian in a pedestrianfeature database.

Although not illustrated, the pedestrian determining unit 240 includes adatabase for storing, as the posture of the sample pedestrian,information associated with front, back, left, and right portions, anupper left half of a body, or an upper right half of a body of thesample pedestrian.

Therefore, the pedestrian determination processor 240 compares thefeatures of the pedestrian candidate extracted by the candidateextraction processor 230 with the posture of the sample pedestrianstored in the database sequentially, for example, in a cascade manner,thereby determining whether the extracted pedestrian candidatecorresponds to a pedestrian.

For example, it is preferable that the postures of the sample pedestrianin the database which are compared with the features of the extractedpedestrian candidate are in the order of the front, back, left, andright portions, the upper left half of the body, and the upper rightpart of the body of the sample pedestrian However, the postures are notnecessarily limited to the order.

The pedestrian tracking processor 250 may track the pedestrianidentified by the pedestrian determination processor 240. For example,the pedestrian tracking processor 250 tracks the pedestrian based on thenumber of feature points of the pedestrian in the ROI. The frontdetection sensor 200 may detect the gaze information of the pedestrianand the distance and the relative speed between the pedestrian and thevehicle by the tracking of the pedestrian tracking processor 250.

FIG. 6 is an exemplified view for identifying gaze information of thefront detection sensor according to an embodiment of the presentdisclosure.

The front detection sensor 200 according to the embodiment of thepresent disclosure may detect the gaze information of the pedestrian andreflect the detected gaze information to the operation of the PDCMS ofthe electronic control unit 400.

In order to detect the gaze information of the pedestrian, the frontdetection sensor 200 sets a horizontal length of the entire face of thepedestrian to be x, a horizontal length from a left face contour to aleft eye to be x₁, a horizontal length from a right face contour to aright eye to be x₂, and a horizontal length between the left eye and theright eye to be x₃ and then detects the gaze information of thepedestrian.

Specifically, the gaze information of the pedestrian corresponds to thefront or the diagonal if |(x₁−x₂)/x|<a, the rear if x₁=x₂=x₃=0, and theside if |(x₁−x₂)/x|≥a or x=0 or x₂=0, in which the a may be selected inthe range between 0.65 and 0.95.

The front detection sensor 200 transmits the detected gaze informationof the pedestrian to the electronic control unit 400. The method forreflecting the gaze information of the pedestrian received by theelectronic control unit 400 to the operation of the PDCMS will bedescribed with reference to FIGS. 10 to 13.

FIG. 7 is a diagram illustrating a concept of a pedestrian moving speed;

Referring to FIG. 7, the front detection sensor 200 may detect adistance between a pedestrian 600 and a vehicle 700 that are movingwithin a driving lane and a moving speed of the pedestrian 600.

For example, if the pedestrian 600 moves from the left to the right withrespect to a front view of the vehicle 700, the pedestrian 600 has anegative (−) moving speed and if the pedestrian 600 moves from the rightto left with respect to the front view of the vehicle 700, thepedestrian 600 has a positive (+) moving speed.

In addition, the front detection sensor 200 may detect the distancebetween the vehicle 700 and the pedestrian 600 moving on the drivinglane of the vehicle.

FIG. 8 is a diagram illustrating an example of a mapping table foractivating a PDCMS function according to the embodiment of the presentdisclosure.

The electronic control unit 400 uses the mapping table to determine therisk of collision of the pedestrian with the vehicle, and furthermore,whether the PDCMS function is operated.

Referring to FIG. 8, the electronic control unit 400 determines theoperation of the PDCMS function based on an initial speed at a boundaryof a driving lane on which the pedestrian moves and an initial speed ofthe vehicle.

Specifically, if an absolute value of the initial speed at the boundaryof the driving lane on which the pedestrian is moving and the initialspeed of the vehicle are in an area in which the PDCMS function isessentially operated at the time of determining whether the PDCMSfunction is operated, the electronic control unit 400 determines thatthe PDCMS function is operated. The operation possible area means thearea in which the Vmin or the Vmax may be adjusted according to theselection of the manufacturer.

For example, if the speed of the vehicle falls below 8.4 m/s (30 km/h)as the Vmin or rises above 16.6 m/s (60 km/h) as the Vmax, then theelectronic control unit 400 may determine that the PDCMS is in thedeactivation state and thus the PDCMS function is not operated.

Further, when the initial speed of the vehicle is between the Vmin andthe Vmax and the absolute value of the initial speed at the boundary ofthe driving lane on which the pedestrian moves is between 0.83 m/s and1.5 m/s, the electronic control unit 400 may determine that the PDCMSfunction is operated.

FIG. 9 is a diagram illustrating an example of the operation of thePDCMS function according to the embodiment of the present disclosure. Avertical axis represents the TTC derived from the distance and therelative speed between the vehicle and the pedestrian and a horizontalaxis represents the operation of the PDCMS function of the vehicle.

The electronic control unit 400 performs the PDCMS operation by stepsaccording to the distance between the vehicle and the pedestrian.

That is, when t₁>t₂>t₃ for t₁, t₂, and t₃ which are different TTCs, ifthe TTC of the vehicle and the pedestrian is t₁, the warning is issuedto the driver through the warning unit 500, if the TTC of the vehicleand the pedestrian is t₂, the vehicle is partially braked, and if theTTC of the vehicle and the pedestrian is t₃, the vehicle is fullybraked.

The warning of the warning unit 500 may include the visual warningthrough the display unit or the audible warning through the speakerunit.

The partial braking means reducing the speed of the vehicle to at leasta predetermined speed or more and the full braking means maximallyreducing the speed of the vehicle.

However, even after the PDCMS function is operated, the driver maymanually operate the brake to perform the maximum possible deceleration.That is, the driver may manually operate the brake to reduce the speedof the vehicle more than the sequential deceleration according to thePDCMS function.

FIGS. 10 to 13 illustrate the method for applying the gaze informationof the pedestrian received from the front-side sensor 200 to theoperation of the PDCMS function for performing the sequentialdeceleration of the electronic control unit 400.

When the pedestrian recognizes the driving direction of the vehicle, thepedestrian is less likely to move to the inside of the route of thevehicle is low and the possibility of collision is reduced accordingly.However, when the gaze direction of the pedestrian is not taken intoconsideration, the gaze of the pedestrian is directed toward the frontof the vehicle, and therefore it is likely to activate the warning andthe operation of the brake due to the unnecessary operation of the PDCMSfunction even in the situation that the pedestrian recognizes thedriving direction of the vehicle and the possibility of collision islow.

The unnecessary operation of the PDCMS function causes the driver toexperience the sense of difference and the discomfort. This is directlyconnected to the commerciality of the vehicle, which is a big problemfor car makers.

Therefore, if the possibility of collision is determined according tothe gaze direction of the pedestrian and the operating time of the PDCMSfunction is controlled, the essential purpose of the PDCMS forprotecting the pedestrian may be achieved and the reduction in the ridecomfort of the driver may be prevented.

FIGS. 10A-10C are exemplified views illustrating the operation of thePDCMS function when the gaze information is the front according to theembodiment of the present disclosure.

As described above with reference to FIG. 6, the horizontal length ofthe entire face of the pedestrian is x, the horizontal length from theleft face contour to the left eye is defined as x₁, the horizontallength from the right face contour to the right eye is defined as x₂,the horizontal length between the left eye and the right eye is definedas x₃.

If the gaze of the pedestrian is directed toward the front of thevehicle, it corresponds to the case where |(x₁−x₂)/x|<a. Therefore, ifit corresponds to |(x₁−x₂)/x|<a based on the lengths of each part of theface of the pedestrian derived from the front detection sensor 200, itmay be determined that the gaze of the pedestrian is directed toward thefront of the vehicle.

In this case, since the vehicle exists within the view of thepedestrian, it is expected that the possibility of collision is lowbecause the pedestrian recognizes the driving direction of the vehicle.Therefore, the electronic control unit 400 may delay the warning of thedriver and the partial braking by a predetermined time from theinitially set times t₁ and t₂ as illustrated in FIG. 9 during theoperating time of the PDCMS function. Alternatively, the electroniccontrol unit 400 may delay all of the warning of the driver, the partialbraking, and the full braking by a predetermined time from all of theinitially set times t₁, t₂, and t₃ as illustrated in FIG. 9 during theoperating time of the PDCMS function.

FIG. 11 is an exemplified view illustrating the operation of the PDCMSfunction when the gaze information is the diagonal according to theembodiment of the present disclosure.

If the gaze of the pedestrian is directed toward the diagonal of thevehicle, it corresponds to the case where |(x₁−x₂)/x|<a like the casewhere the gaze of the pedestrian is the front of the vehicle. Therefore,if the gaze of the pedestrian corresponds to |(x₁−x₂)/x|<a based on thelengths of each part of the face of the pedestrian derived from thefront detection sensor 200, it may be determined that the gaze of thepedestrian is directed toward the diagonal of the vehicle.

In this case, since the vehicle exists within the view of thepedestrian, it is expected that the possibility of collision is lowbecause the pedestrian recognizes the driving direction of the vehicle.Therefore, the electronic control unit 400 may delay the warning of thedriver and the partial braking by a predetermined time from theinitially set times t₁ and t₂ as illustrated in FIG. 9 during theoperating time of the PDCMS function. Alternatively, the electroniccontrol unit 400 may delay all of the warning of the driver, the partialbraking, and the full braking by a predetermined time from all of theinitially set times t₁, t₂, and t₃ as illustrated in FIG. 9 during theoperating time of the PDCMS function.

FIG. 12 is an exemplified view illustrating the operation of the PDCMSfunction when the gaze information is the rear according to theembodiment of the present disclosure.

If the gaze direction of the pedestrian is in the same direction as thedriving direction of the vehicle, that is, if the vehicle is drivingtoward the rear of the pedestrian, it corresponds to x₁=x₂=x₃=0.Therefore, if it corresponds to |(x₁−x₂)/x|<a based on the lengths ofeach part of the face of the pedestrian derived from the front detectionsensor 200, it may be determined that the gaze direction of thepedestrian is the same as the driving direction of the vehicle, that is,the vehicle is driving toward the rear of the vehicle.

In this case, since the vehicle exists within the view of thepedestrian, it is expected that the possibility of collision is highbecause the pedestrian does not recognize the driving direction of thevehicle. Therefore, the electronic control unit 400 may not delay theoperating time of the PDCMS function from the initially set times t₁,t₂, and t₃ as illustrated in FIG. 9.

FIG. 13 is an exemplified view illustrating the operation of the PDCMSfunction when the gaze information is the side according to theembodiment of the present disclosure.

If the gaze direction of the pedestrian forms about 90° with respect tothe driving direction of the vehicle, that is, if the vehicle is drivingtoward the side of the pedestrian, it corresponds to the case where|(x₁−x₂)/x|≥a or x₁=0 or x₂=0. Therefore, if it corresponds to|(x₁−x₂)/x|≥a or x₁=0 or x₂=0 based on the lengths of each part of theface of the pedestrian derived from the front detection sensor 200, itmay be determined that the gaze direction of the pedestrian forms about90° with respect to the driving direction of the vehicle, that is, thevehicle is driving toward the side of the vehicle.

In this case, since the possibility that the vehicle does not existwithin the view of the pedestrian is significant, it is expected thatthe possibility of collision is high because the pedestrian does notrecognize the driving direction of the vehicle. Therefore, theelectronic control unit 400 may not delay the operating time of thePDCMS function from the initially set times t₁, t₂, and t₃ asillustrated in FIG. 9.

FIG. 14 is a flow chart illustrating a flow of a method for activating aPDCMS function according to an embodiment of the present disclosure.

Referring to FIG. 14, a method for activating a pedestrian detection andcollision mitigation system (PDCMS) function according to an embodimentof the present disclosure includes: acquiring an image in front of avehicle (S100); setting a region of interest (ROI) corresponding to asize of an object in the image (S200); extracting a pedestrian candidatebased on an operation of the object in the set ROI (S300); including adatabase and comparing the extracted pedestrian candidate with a postureof a sample pedestrian in the database to identify whether the extractedpedestrian candidate corresponds to a pedestrian on a driving lane ofthe vehicle (S400); detecting, by the front detection sensor, gazeinformation of the pedestrian and a distance and a relative speedbetween the pedestrian and the vehicle (S500); detecting, by a vehiclesensor, at least one of a speed, an acceleration, a steering angle, asteering angular velocity, and a pressure of a master cylinder of thevehicle (S600); and operating the PDCMS function based on theinformation detected by the front detection sensor and the vehiclesensor (S700).

In the acquiring of the image in front of the vehicle (S100), the imageis acquired by the camera photographing the front of the vehicle.

In the setting of the region of interest (ROI) corresponding to the sizeof the object in the image (S200), the ROI corresponding to the size ofthe object in the image in front of the vehicle acquired in the stepS100 is set. That is, the ROI setting processor 220 uses the size of theimage, the installation environment information of the camera, thespecifications of the camera, and the actual size of the pedestrian todetermine the floor position of the pedestrian area in the image.Therefore, since the pedestrian may be identified based on the number ofpixels, the ROI setting processor 220 may set the minimum and maximumsize of the pedestrian from the floor position of the pedestrian area tobe an appropriate ROI.

In the extracting of the pedestrian candidate based on the operation ofthe object in the set ROI (S300), the pedestrian candidate is extractedbased on the operation of the object having the set ROI in the image.

The including of the database and the comparing of the extractedpedestrian candidate with the posture of the sample pedestrian in thedatabase to identify whether the extracted pedestrian candidatecorresponds to the pedestrian on the driving lane of the vehicle (S400),the features of the extracted pedestrian candidate are compared with theposture of the sample pedestrian stored in the database sequentially,for example, in the cascade manner to determine whether the extractedpedestrian candidate corresponds to the pedestrian.

In the detecting of, by the front detection sensor, the gaze informationof the pedestrian and the distance and the relative speed between thepedestrian and the vehicle (S500), the gaze information of thepedestrian and the distance and the relative speed between thepedestrian and the vehicle are detected by tracking the pedestrian basedon the number of feature points of the pedestrian in the ROI.

In the detecting of, by the vehicle sensor, the vehicle informationincluding at least one of the speed, the acceleration, the steeringangle, the steering angular velocity, and the pressure of the mastercylinder of the vehicle (S600), the vehicle sensor measures the RPM ofthe vehicle wheel based on the vehicle engine and calculates the RPM ofthe vehicle based on the known circumference of the wheel and themeasured RPM and time to detect the information on the drivingconditions of the vehicle such as the acceleration, the steering angle,the steering angular velocity, and the pressure of the master cylinder.

In the operating of the PDCMS function based on the information detectedby the front detection sensor and the vehicle sensor (S700), it isdetermined whether the conditions that the PDCMS function may beoperated are satisfied by combining the pedestrian state and the vehiclestate. For example, it is determined whether the conditions that thePDCMS function on the mapping table may be operated are satisfied basedon the combination of the pedestrian state and the vehicle state. Thatis, if it is determined that an obstacle is a pedestrian, the risk ofcollision of the pedestrian with the vehicle is determined on themapping table using the current position of the pedestrian, the currentposition of the vehicle, and the speed information of the vehicle.

Further, if it is determined that the pedestrian state and the vehiclestate satisfy the conditions that the PDCMS function may be started, thePDCMS function of the vehicle is operated. The PDCMS function includesthe activation of the operation of the warning unit that is operated toinform the driver of the collision of the pedestrian with the vehicleand the operation of the brake regardless of whether the driver operatesthe brake. The activation of the operation of the warning unit and theactivation of the operation of the brake are performed in order of theoperation of the warning unit, the partial braking of the vehicle, andthe full braking of the vehicle.

Meanwhile, it should be understood that the PDCMS was described as anexample for convenience of description in the present specification. Asdescribed above, it should be understood that the PDCMS is only one ofseveral ADAS functions, and that the PDCMS implementations presented bythe present disclosure may also be used to implement other ADASfunctions involved. For example, the system presented by the presentdisclosure may be applied to implement one or a combination of ones ofthe ADAS functions such as the PDCMS, a lane change decision aid system(LCDAS), a land departure warning system (LDWS), an adaptive cruisecontrol (ACC), a lane keeping assistance system (LKAS), a road boundarydeparture prevention system (RBDPS), a curve speed warning system(CSWS), a forward vehicle collision warning system (FVCWS), and lowspeed following (LSF).

The apparatus for activating a PDCMS according to the embodiment of thepresent disclosure may accurately determine the pedestrian based on theposture information of the pedestrian in the image in front of thevehicle.

Further, the vehicle braking control may be performed while beingdelayed by the predetermined time only when the pedestrian may recognizethe vehicle based on the gaze information of the pedestrian.

Therefore, the apparatus for activating a PDCMS according to theembodiment of the present disclosure may more accurately detect thepedestrian to effectively protect the pedestrian and optimize thevehicle braking control unnecessarily frequently performed to resolvethe sense of difference and discomfort of the driver, thereby improvingthe commerciality of the vehicle.

The foregoing includes examples of one or more embodiments. Of course,all possible combinations of components or methods for the purpose ofdescribing the embodiments described above are not described, but thoseskilled in the art may recognize that many combinations andsubstitutions of various embodiments are possible. Accordingly, thedescribed embodiments are intended to embrace all the alternatives,modifications and variations that fall within the spirit and scope ofthe appended claims. Moreover, in connection with the extent that theterm “include” in the detailed description or the appended claims isused, the term are intended to be inclusive in a manner similar to“consisting” as interpreted when the term “configured” is used as atransitional word in the appended claim.

What is claimed is:
 1. An apparatus for activating a pedestriandetection and collision mitigation system (PDCMS) of a vehicle, theapparatus comprising: a front detection sensor detecting a presence of apedestrian on a driving lane of the vehicle, gaze information of thepedestrian, and a distance and a relative speed between the pedestrianand the vehicle; a vehicle sensor detecting at least any one of a speed,an acceleration, a steering angle, a steering angular velocity, or apressure of a master cylinder of the vehicle; an electronic control unitactivating a PDCMS function based on information detected by the frontdetection sensor and the vehicle sensor; and a warning unit operated toinform a driver of a collision of the pedestrian with the vehicle by acontrol of the electronic control unit, wherein the front detectionsensor includes: an image input processor acquiring an image in front ofthe vehicle; an ROI setting processor setting a region of interest (ROI)corresponding to a size of an object in the image; a candidateextraction processor extracting a pedestrian candidate based on anoperation of the object in the set ROI; and a pedestrian determinationprocessor including a database and comparing the extracted pedestriancandidate with a posture of a sample pedestrian in the database toidentify whether the extracted pedestrian candidate corresponds to thepedestrian, wherein the PDCMS function includes an activation of anoperation of the warning unit and an activation of an operation of abrake regardless of whether the driver operates the brake, and whereinthe activation of the operation of the warning unit and the activationof the operation of the brake are performed in order of the operation ofthe warning unit, a partial braking of the vehicle, and a full brakingof the vehicle.
 2. The apparatus of claim 1, wherein the pedestriandetermination processor stores, as the posture of the sample pedestrian,information associated with at least one of front, back, left, and rightportions, an upper left half of a body, and an upper right half of abody of the sample pedestrian in the database.
 3. The apparatus of claim2, wherein the pedestrian determination unit compares the extractedpedestrian candidate with the front, back, left, and right portions, theupper left half of the body, or the upper right half of the body of thesample pedestrian in the database in a cascade manner.
 4. The apparatusof claim 1, wherein the front detection sensor further includes apedestrian tracking processor tracking the pedestrian identified by thepedestrian determination processor.
 5. The apparatus of claim 4, whereinthe pedestrian tracking processor tracks the pedestrian based on anumber of feature points of the pedestrian in the set ROI.
 6. Theapparatus of claim 1, wherein when a horizontal length of the entireface of the pedestrian is defined as x, a horizontal length from a leftface contour to a left eye of the pedestrian is defined as x₁, ahorizontal length from a right face contour to a right eye of thepedestrian is defined as x₂, and a horizontal length between the lefteye and the right eye of the pedestrian is defined as x₃, the gazeinformation of the pedestrian corresponds to a front or a diagonal if|(x₁−x₂)/x|<a, a rear if x₁=x₂=x₃=0, and a side if |(x₁−x₂)/x|>a or x₁=0or x₂=0, and the a is selected in a range between 0.65 and 0.95.
 7. Theapparatus of claim 6, wherein the electronic control unit performs theactivation of the operation of the warning unit and the activation ofthe operation of the brake by delaying the activation of the operationof the warning unit and the activation of the operation of the brake bya predetermined time when the gaze information of the pedestrian is thefront or the diagonal, compared to when the gaze information of thepedestrian is the rear or the side.
 8. The apparatus of claim 1, whereinthe vehicle sensor further includes at least any one of a rain sensor, atemperature sensor, and an illumination sensor.
 9. The apparatus ofclaim 1, wherein the electronic control unit performs the activation ofthe operation of the brake so that the speed of the vehicle is reducedto at least a predetermined speed or more from time when the operationof the brake is activated to time when the collision of the pedestrianwith the vehicle occurs.
 10. The apparatus of claim 1, wherein theelectronic control unit permits the driver to operate the brake for amaximum possible deceleration even after the activation of the operationof the brake starts.
 11. The apparatus of claim 1, wherein theelectronic control unit controls the warning unit to inform the driverthat the PDCMS function is in an available state.
 12. The apparatus ofclaim 1, wherein the warning unit includes a display visually informingthe collision of the pedestrian with the vehicle or a speaker audiblyinforming the collision of the pedestrian with the vehicle.
 13. Theapparatus of claim 1, wherein the PDCMS function further includes anoperation of a rear brake lamp.
 14. The apparatus of claim 1, whereinthe PDCMS function further includes an operation of an electricalstability control (ESC).
 15. A method for activating a pedestriandetection and collision mitigation system (PDCMS) of a vehicle, themethod comprising: acquiring, by a front detection sensor, an image infront of the vehicle; setting, by the front detection sensor, a regionof interest (ROI) corresponding to a size of an object in the image;extracting, by the front detection sensor, a pedestrian candidate basedon an operation of the object in the set ROI; including, by the frontdetection sensor, a database and comparing the extracted pedestriancandidate with a posture of a sample pedestrian in the database toidentify whether the extracted pedestrian candidate corresponds to thepedestrian on a driving lane of the vehicle, and detecting, by the frontdetection sensor, gaze information of the pedestrian and a distance anda relative speed between the pedestrian and the vehicle; detecting, by avehicle sensor, at least one of a speed, an acceleration, a steeringangle, a steering angular velocity, or a pressure of a master cylinderof the vehicle; and activating, by an electronic control unit, a PDCMSfunction based on information detected by the front detection sensor andthe vehicle sensor, wherein the PDCMS function includes an activation ofan operation of a warning unit operating to inform a driver of acollision of the pedestrian with the vehicle and an activation of anoperation of a brake regardless of whether the driver operates thebrake, and the activation of the operation of the warning unit and theactivation of the operation of the brake are performed in order of theoperation of the warning unit, a partial braking of the vehicle, and afull braking of the vehicle.